1. Field of the Invention
The present invention relates to a computer based browsing computer program product, system and method for browsing through documents stored in a computer. Specifically, the present invention facilitates the browsing of a document stored in a computer in such a way that it allows a rapid and holistic view of what contents are present and the organization of the material in the document, as well as rapid access to the items in the document involved. The document-browsing system in the present invention is also generalized to provide a novel human-computer interface, not just for browsing documents but also for interacting with and operating a computer to achieve the various functionalities normally carried out on the computer.
2. Description of the Related Art
Currently, the method for viewing documents stored in a computer is through a computer monitor screen. For documents that are longer/larger than can be contained within one screen, a way is provided to scroll the document up and down or to jump to a particular point in the document (through the use of, for example, a xe2x80x9cmousexe2x80x9dcoupled with scroll bars at the edges of the document displayed on the screen or through the use of hypertext links).
However, most people do not find this a particularly convenient way to view a document, as compared with a book. That this is true is evidenced in the fact that despite the trend of increased computer usage, there is not only no corresponding reduction in paper use, but an increase in paper quantity is required to print what is conveniently stored in a computer that can be seemingly easily and flexibly manipulated for viewing. Sometimes the printed version of the information involved is necessary, like in the case of printing a picture on a sheet of paper to be pasted on some surface (e.g., a wall), or in the case of the convenience of having a relatively light, hardy paperback book for reading in almost any possible situation and location. Reading textual material on the computer screen for a prolonged period of time also leads to visual fatigue. But there are times when a reader does not need to read for a prolonged period of time and is quite happy to sit at a desk in front of a desktop computer or handle a notebook computer away from the desk, and yet the reader would still prefer a printed version of the document in a hand-held format. This is especially true in the case of looking through manuals, including software manuals, to understand how to operate a device or software application. The irony is that computer software packages (included in computer readable media, such as optical disks, magnetic disks, semiconductor memory, etc.) usually come with thick and heavy manuals containing information which can easily be stored on a light-weight CD-ROM, and even if the manuals are sometimes dispensed in the form of a CD-ROM, more often than not people would print them out into a hard-copy format in order to facilitate reading.
What is involved when one interacts with the printed material in a book is a subtle and complicated process. To start with, material in a book is presented in a sequential order, with a continuity of material from page to page, and there is also a hierarchical structure in the material presented (as the material is organized into chapters, sections, subsections, etc.) because ideas in the material are related to each other in some kind of conceptual hierarchy. The human perceptual system inputs data in a sequential manner, and after a book is read from the beginning to the end in a sequential fashion, the brain then recreates the conceptual hierarchy after viewing the material involved. However, very often one does not read a book (or input the material involved) from the beginning to the end because (a) one wants to have an overview of the material present; (b) one is searching for something of interest to him/her; or (c) one is interested in reading only portions of the book (in the case of, say, reading the manual to understand how to operate something). In these cases, one browses through the subject book to find the material of unique interest to that reader.
Two basic things are achieved in the browsing process. First, the browser has a glimpse of what are the contents of the book document. Second, the browser has an idea of approximately where the items of interest are so that the browser can (a) return to look for them later when needed, and (b) have an understanding of the relationships between the material currently being viewed and other material (i.e., an understanding of the hierarchical structure involved). When browsing a book document, many finger-operations are required of the browser in order to flip through the pages and, together with the inherent sequential order imposed by the pages, very quickly allow the browser to have an understanding of the nature, location and organization of the material involved.
In the process of browsing through a book, one can perform the following operations:
(a) flip through the pages at varying speeds depending on the level of detail at which one wishes to view the material in the book;
(b) jump to the approximate location of the item of interest;
(c) change the direction of flipping (forward or backward) very rapidly because
(i) one would like to compare and contrast material on different pages,
(ii) after jumping to an approximate location of some items of interest one would like to find their exact locations, or
(iii) one is unsure of where the item of interest is and is in the process of searching for it; and
(d) mark the locations of some pages of interest that one may want to later return.
All these operations are performed very rapidly with the fingers interacting with the flipping pages and with minimal unnecessary movements of the fingers and hands. Interestingly, a book/magazine with soft and flexible pages is harder to handle because more finger and hand movements are needed to browse through it, while books with stiff pages can be browsed with almost no movement of the hand.
In currently available methods of browsing through documents stored in a computer, e.g., the use of a mouse combined with scroll bars and buttons on the computer screen, more movements of the hands are necessary to effect the various operations described above. Also, fine control of the hand or fingers (depending on whether the mouse uses hand movement to move the cursor on the screen or finger movement like in the case of a track ball) is necessary to position the cursor on the screen at the required places. The process is both lengthy and clumsy. The lengthiness of the process taxes the human short term memory""s ability to remember items encountered in the recent past for the purpose of establishing the relationships between items and the clumsiness of the process creates distraction and interferes with the short term memory process, a well known effect in perceptual psychology. These destroy the ability of the reader to form an idea of the contents and structurexe2x80x94i.e., a holistic viewxe2x80x94of the document involved.
Because material in a book is organized into pages, it allows the reader to quantify approximately the amount of coverage given to each topicxe2x80x94e.g., three-quarters of a page, three-and-a-half pages, etc. In addition, page-oriented organization together with the fact that the pages are joined together in a sequential order with a continuity of material from page to page enhance the ability of the reader to better remember the location of various portions of the material involved (e.g., roughly how many pages from the pages currently in view). Also, unlike the process of scrolling through a document on a screen like what is normally done in a word-processor, wherein the contents become a blur and reading is impossible, when one moves through the material in a book through flipping, one is still able to read at least the approximate contents, if not the details. This, together with the page-oriented organization that enhances the user""s impression of the location of material viewed, allow the user to form an impression of the organization of the material in the document.
Another method used for presenting information on the computer screen is flashing one page of information after another. This method eliminates the problem of blurred images but it also eliminates the continuity between different portions of the document. In turn, the lack of continuity eliminates the reader""s ability to form impressions of the approximate absolute locations of these portions of the material involved and hence the impressions of the relationships between these portions of the material. As recognized by the present inventor, only the flipping method normally present in a book (e.g., a hardcover book) is able to provide both continuity and readable pages in view. It is due to these features that a person browsing through a book can acquire a good understanding of its contents, the location of specific items and organization of the material, and hence a holistic view of the book involved.
As recognized by the present inventor, it is because of the reasons set forth above that people still prefer to read a book in their hands rather than a document image displayed on a computer screen using currently available methods.
U.S. Pat. No. 5,467,102 (Kuno et al.) discloses a device for document viewing that consists of two display screens. One of the purposes of using two display screens is to allow the user to display two different pages from the document so that they can be read side-by-side (e.g., a diagram and its textual explanation). Another purpose is to allow a large picture to be displayed simultaneously on both pages. The Kuno et al. device allows users to change the speed of movement through the document through a pressure sensorxe2x80x94the more pressure applied, the faster the pages in the document are moved through. The Kuno et al. device also allows the document to be viewed in the forward or backward direction by pressing on a forward sensor area or a reverse sensor area respectively. One can also select a page to jump to by pressing on an icon displayed on the screen. However the Kuno et al. device still does not provide the same convenience as browsing through a book, primarily because when switching between the operations for different controlsxe2x80x94the speed of movement through the document, the change of direction of viewing, and the jumping to different parts of the documentxe2x80x94there are a lot more hand and finger movements than is the case in manipulating a physical book. Moreover, the Kuno et al. device is a specialized, relatively costly device with sensors and hardware built onto two display screens, whereas the present inventor recognizes that a lower cost and more practical device would be one that adds modularly to the existing computer system.
Currently, there are also computer mice that can eliminate the above-mentioned problem of positioning cursor on the computer screen with a conventional mouse (i.e., fine control of the hand or fingers is needed). These mice allow the user to specify xe2x80x9chot locationsxe2x80x9d on the screen on which the cursor xe2x80x9chomes ontoxe2x80x9d with less fine control than conventional mice. Furthermore, a subset of these mice can generate xe2x80x9cvertical onlyxe2x80x9d or xe2x80x9chorizontal onlyxe2x80x9d movement of the cursor so that the scrolling process requires less fine control of the muscle than is required with a conventional mouse. These mice eliminate some, but not all, of the problems associated with the conventional methods of computer input as far as computer-based document browsing is concerned.
In U.S. Pat. No. 5,417,575 (1995) McTaggart discloses an electronic book that comprises laminated sheets bound together in the form of a book. On each of these sheets, printed material is arranged on the top layer and below that layer is an electronic backdrop containing thin light-emitting diodes (LED""s) and pressure sensitive switches affixed onto a backing sheet. The LED""s generate visual signals that can be seen through the top layer for the purpose of highlighting parts of the printed material. The pressure switches, positioned under certain items in the printed material, are for the purpose of sensing the user""s selection of those items. A speaker is also provided on the book to generate audio signals for explaining the text or giving the user audio feedback. Contact or photo-sensitive switches are also embedded in the pages to allow the electronic circuits to know which pages are currently being viewed, so that the appropriate audio and visual signals can be generated. Even though this apparatus is in a form that allows a person to handle it like handling a typical book, with visual and audio enhancements of the printed material as well as facilities that accept the user""s feedback, it is basically a hard-wired device that is not reprogrammable and different hardware has to be configured for books with different contents. No provision is available for downloading document files from a computer for display on the electronic book nor is the electronic book able to display any arbitrary document file. This device is hence not suitable for browsing through documents stored in a computer.
Therefore, the present inventor has identified there exists a need for a low-cost, modular device that can be connected to existing computer systems and that, through the computer executing a set of operations stored on a computer-readable medium, can effect rapidly the necessary browsing operations and together with a preferred mode of moving through a documentxe2x80x94e.g., the flipping modexe2x80x94that permits easy, effective computer-based document browsing that approaches that of browsing through a book. A feature of such a system is that it would exhibit functional attributes like those of a paper-book (i.e., a physical book such as a paper-back book).
The method of browsing a physical book can also benefit another aspect of human-computer interaction. In 1997, the most popular type of human-computer interfacexe2x80x94the WINDOWS interface (e.g., MICROSOFT Corporation""s WINDOWS 95 (copyright) or APPLE Computer""s MAC operating system, OS)xe2x80x94also suffers from many of the problems associated with reading documents on a computer. Basically, there are two situations when xe2x80x9cwindowsxe2x80x9d are created on the computer screen. The first is when the user needs to look into the contents of xe2x80x9cdirectoriesxe2x80x9d containing either other directories or computer programs and data files. In this situation, what are displayed in a window when it is opened are the contents of a directory. This way, a user can navigate among the directories organized in a tree structure and resident on the computer hard-disks to locate programs (often called xe2x80x9capplicationsxe2x80x9d) and data. The other is when a user activates a program, and the program either generates some output to be displayed on the computer screen or is waiting to accept some user input (usually either a command to the program or data to be fed into the program). In the WINDOWS interface, these output and input are presented and accepted through a xe2x80x9cwindowxe2x80x9d, referred to as the working window of a program.
In the case of the use of multiple windows to display the contents of directories in the process of searching for a certain item (either a directory, a program or a data file), if one knows exactly under which directory an item of interest is placed, the tree-oriented structure present in this kind of system can greatly facilitate the search for the item. However, if one does not know where the item is placed, one would need to carry out a large number of operations opening/entering and closing/exiting directories, often across and up and down many levels of the directory structure, and this is carried out with a large number of, and often tiring, operations involving clicking and dragging the mouse (or other pointing device).
The situation is compounded by the fact that often the contents of a directory cannot fit entirely within a window. When that happens, a mouse-cum-scroll-bar method, much like that used for browsing through documents on a computer, is available for use to reveal the hidden parts of the directory involved. The situation is similar to viewing documents on computers, where the process of entering, exiting and scrolling up and down directory windows to search for items of interest demands a large number of finger and hand operations and movement. Also, the flashing in and out of view of these directory windows results in a discrete, non-continuous, and random sequence of images. This results in a poor idea of items"" location in the entire directory structure as well as the inability to form a holistic view of the entire collection of program and data files resident on the computer""s hard-disk.
In the case of computer programs using multiple windows to interact with a user, because each opened window tends to occlude other windows already present, it is often the case that in the process of accomplishing a certain task (e.g., creating a multimedia document), a large and often confusing sequence of windows has to be opened, closed or moved aside, each window representing an activation of a program to accomplish some subtasks (e.g., extracting some text from a text file, extracting a sub-part of an image from an image file, carrying out some processing on the images thus obtained, xe2x80x9cpastingxe2x80x9d all these items together in yet another program, etc.) Again a large number of finger and hand operations and movement is required to carry out these operations through the use of a mouse. Furthermore, because the various windows thus opened in this process do not have any fixed locations, and their relative locations keep changing, an excessive amount of mouse operations is often needed just to locate these opened windows or put them away for later use.
The WINDOWS interface derives from the desktop metaphor, in which the computer screen is likened to the desktop, and the windows are thus analogous to items that one normally handles on a desktop. While the desktop interface is a large improvement over the previous kind of command line interface (e.g., MICROSOFT""s DOS(copyright)), primarily because of its user-friendly graphical interface and menu driven input, it suffers from the same problem of the desktop: when the items to be handled are many, the desktop tends to become very messy, and one often has to keep pushing aside many items to look for other items.
As a practical matter, when a large number of sheets of papers are handled by a person, either to collect information on various things or to scribble down ideas, one often joins (perhaps by a staple or binder clip) these sheets of paper into a temporary xe2x80x9cbookxe2x80x9d, or one uses a scrap book with blank pages on which the user may scribble-down ideas. However, the working items, once bound together, are not necessarily related to one another, and thus, are different than a paper book in which the various portions of material are often related to one another. This organization of these sheets of paper allows one to obtain the usual advantages associated with a book, namely, the items of interest acquire absolute locations, and together with the finger/hand operations one can normally apply to a book to search for information, the items of interest can easily be located. Because the pages thus organized in a book can be quickly and easily located and viewed, the effect is tantamount to having multiple pages in view almost simultaneously, and hence it will alleviate the problems associated with the occlusion of pages by other pages or windows by other windows.
Thus, the present inventor has identified that there is a need for a higher level of organization built onto the currently available WINDOWS interfacexe2x80x94a xe2x80x9cBooksxe2x80x9dinterface- that can be used in conjunction with the above-mentioned low-cost, modular device that can alleviate many of the problems associated with the WINDOWS interface in order to facilitate human-computer interaction.
In view of the aforementioned short-comings of presently available schemes for browsing through documents stored in a computer, one objective of the present invention is to overcome these short-comings of conventional systems, computer-readable media, and methods.
Another object is to provide a browsing device that exploits the use of finger operations normally involved in browsing through a book, namely, the change of speed of movement through the document involved, the change of direction of movement through the document, the jumping to other portions of the document and the bookmarking of pages (e.g., when a page is bookmarked, it can be returned/jumped to later very quickly by using the controls operated by the fingers).
Another object of the invention is that the positioning, arrangement, and operability of the controls for the above-mentioned operations performed by the fingers are such that they allow almost no movement of the hand and minimal movements of the fingers, thus maximizing the ease of browsing through the stored document by capitalizing on the dexterity of the human fingers to operate these controls.
A further object of the invention is to provide a low cost, modular browsing device that can be easily attached to existing computer systems much like how a mouse attaches to a computer system.
Still a further objective is to provide a reconfigurable construct for the browsing device so that it can be (a) configured into a hand-held controller; (b) attached to the sides of existing computer screens; (c) configured to cooperate with a mouse so that there is no need to move the user""s hand(s) when switching between browser-related operations and mouse-related operations; and (d) configured to cooperate with a gyro-mouse so that the entire assembly can be used in the absence of a table top, where the particular construct is selected by the user depending on the user""s preference.
Another object of the invention is to provide a computer-readable medium that holds software that when executed by a computer displays, on the computer screen, the document to be browsed by way of browsing commands generated by the user actuating the browsing device. The display is in the form of a book format image that shows the thicknesses of material in the document before and after the currently viewed material, showing of the bookmarks, and showing, on the thicknesses, of the location of the pages that would be jumped to at any given time if jumping were to be effected.
Another object of the invention is to provide a system, computer-based process, and method that permit different organizations of material in the document and corresponding display formats, including: (a) organized into pages that can be flipped through page by page from right to left or vice versa, much like what happens when one flips through a book; (b) organized into pages that can be flipped through page by page from bottom to top or vice versa, much like what happens when one flips through a notepad; (c) organized into pages that can be slid through page by page from right to left or vice versa, much like what happens when one views a microfilm; (d) organized into pages that can be flashed one after another; and (e) organized into pages that can be scrolled through, with no distinct page boundaries, much like what is normally done in a computer implemented word-processor application stored on a computer-readable medium. The first three methods when used together with the aforementioned browsing device result in a browsing process that most resembles that of browsing through a book, a notepad, or a micro-film. The last two methods do not precisely emulate the book-browsing process but can still benefit from the rapidity of finger control effected on the browsing device.
Another object of the invention is to provide a computer based mechanism that implements simultaneous multiple indexing in conjunction with the inventive browsing system. When a keyword of interest is encountered during the reading of the document, the user selects the keyword using a cursor on the screen in conjunction with a pointing device such as a mouse, like what is normally done in current computer systems, or using his/her finger in conjunction with a touch or pressure sensing screen, or other methods. In response, the pages on which explanations or other issues related to this keyword reside are bookmarked by the inventive system, after which the user can quickly flip to one of these pages using the mechanisms provided in the browsing system.
Another object of the invention is to use the above-described browsing facility in conjunction with other software, storable on computer readable media, that is configured to reorganize the material in the document involved to facilitate browsing/viewingxe2x80x94for example, the positioning of material for comparison side by side on the pages currently being viewed.
Another object of the invention is to use the above-mentioned browsing facility in conjunction with software, storable on computer readable media, that can highlight selected portions of the material or annotate on the pages in the document involved to facilitate browsing/viewing/reading.
Yet another object of the invention is to provide the above-mentioned browsing facility to a computer configured to execute word processing software so as to facilitate the entering, processing, and viewing/browsing of material in conjunction with the word processing software.
Another object of the invention is to provide the above-mentioned browsing facility to any software, storable in a computer-readable medium, in which information cannot be fitted within one computer screen for viewing or manipulating.
Yet another object of the invention is to add a new metaphor mechanismxe2x80x94the xe2x80x9clibrary metaphorxe2x80x9dxe2x80x94to the computer operating system""s human interface in which information on the computer screen which is traditionally presented in the form of windows is now be presented in the form of xe2x80x9cbooksxe2x80x9d.
These and other objects are achieved by providing a processor-based browsing device, system, computer-readable media, and method for browsing through a document that includes, a top surface, a sensor surface and a bottom surface. A sensor area on the sensor surface detects the position of a finger along one direction and the force of the finger on the sensor area. The browsing device also includes four buttons/on-off switches on the bottom surface, and four additional buttons/on-off switches on the said top surface, each of which is operated by the fingers. An electrical circuit converts the force and position of the finger on the sensor area as detected by the force and position sensors on that area into electrical forms and outputs them. The electrical circuit also outputs the on/off states of the four buttons/on-off switches on the bottom surface, and four additional buttons/on-off switches on the said top surface. The device connects to a computer configured to implement in software the control, sense, and display mechanisms consistent with the users activity with the browsing device.